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Abstract 



Recently, a number of evaluations of private school voucher programs in the United States have 
reported achievement gains for voucher users, especially if they are African American. These 
studies tend to be structured as Randomized Field Trials (RFTs), where participants are assigned 
to treatment (offered a voucher) and control (not offered a voucher) groups by lottery. A major 
advantage of RFTs is that the randomization process controls for a number of factors, 
measurable and uiuneasurable, which otherwise might confound the assessment of voucher 
effects. A major shortcoming of RFTs is that they tend to be “black-box” evaluations that tell 
policy analysts precious little about why or how a policy intervention yields benefits 
downstream. Here we use data from the second year RFT of the Washington, D.C. privately- 
funded voucher program, supplemented by information obtained from the schools that 
participating students attended, in an effort to identify what school features or practices might be 
boosting the achievement of voucher students. This preliminary analysis suggests that especially 
dedicated teachers, a higher proportion of white and higher-income students, and more 
demanding homework assigmnents may be the characteristics of private schools that increase the 
academic achievement of irmer-city school voucher users. Additional analyses are recommended 
before firm conclusions are drawn regarding what happens inside the black box of school 
voucher experiments. 
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I. Introduction 



Recently a niunber of major studies have been conducted on the effects of school 
vouchers on the educational achievement of students (Howell et al 2002, Greene 2001, Peterson 
et al 1999). These studies tend to identify positive academic effects of vouchers on the African- 
American students who use them to switch from public to private school. The studies are 
designed as Randomized Field Trials (RFTs), with families assigned to treatment and control 
groups by lottery. The randomization process tends to control for all other factors, besides the 
treatment, that otherwise might confound the analysis. Therefore, analysts can ascribe to the 
treatment responsibility for any post-intervention differences between the students who received 
vouchers and the control group who did not. Economist Carolyn Hoxby has described this 
methodological approach as the “gold standard” for research on educational interventions, such 
as vouchers, that otherwise would be vulnerable to self-selection bias (Hoxby 2000a). 

In spite of the advantages of these RFTs for correctly identifying the independent effects 
of school voucher interventions, they have at least one significant shortcoming: by themselves 
they do not indicate why vouchers improve the educational performance of the students who use 
them. Randomized Field Trials are classic “black-box” analyses, where a condition is measured 
pre-test, the treatment is administered to one randomly determined group but not another group, 
and a post-test is administered. Participants are baseline tested, one group is permitted to enter 
the black box, and the groups are re-tested after the treatment group emerges from the black box. 
What happens inside the box in order to generate post-test differences in the two groups is not 
the focus of such studies. 

This agnosticism regarding the proximate causes of school voucher gains prevents the 
RFTs from being maximally informative regarding policy reforms. If voucher students learn 
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more simply because their private schools have smaller class sizes, more elaborate facilities, or 
assign more homework, then such beneficial traits could be adopted by the public schools that 
serve the low-income inner-city populations who are eligible for vouchers, thus obviating the 
need for them to seek a private school education. Moreover, if voucher gains are simply due to 
the benefits that low-income students receive from interacting with higher income or more 
racially diverse peers, then voucher effects might diminish as more low-income African- 
American students take advantage of such programs (Fisk and Ladd 2000). There is simply a 
finite pool of higher income peers available to mix with lower income students, whether one 
lives in New Zeeland or Lake Wobegone. 

As a result, it is critically important for us to try to discern what is happening inside the 
black box of voucher interventions that is making a positive difference for low-income iimer-city 
students. This paper puts forward the initial baby-steps for such a journey. Using the data from 
the second year evaluation of the Washington Scholarship Fimd school voucher program, we 
attempt to identify one or more particular characteristics of schools that represent proximate 
causes for the greater academic achievement of voucher students. Our analysis is essentially 
exploratory and our findings are highly preliminary. They suggest that there is no single silver 
bullet characteristic that individually explains voucher gains. However, it appears that greater 
school resources, smaller schools, smaller class sizes, order and discipline, and a stronger sense 
of commimity do not explain the voucher gains. The best candidates for factors that are driving 
the educational improvement of voucher users are more racially diverse and economically 
advantaged peers, more homework, and dedicated teachers. More research will be necessary in 
order to confirm that these initial findings are reliable. 
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II. Theory 

The established theories regarding which school characteristics improve the performance 
of low-income inner-city students are both rich and varied. They include claims that school 
resources, size, commxmity, order and discipline, high expectations, and student body 
demographics are the reasons why some disadvantaged urban students achieve more than others 
do. We will consider these theories in turn. 

A. Resources 

Entire rain forests have been felled in the debate over whether or not increased resources 
measurably contribute to positive educational outcomes. Politicians certainly behave as if they 
believe that more resources purchase better education, as per-pupil spending on education has 
more than quintupled, in real terms, over the past fifty years (Brandi 1998, 2-3). Moreover, 
educational equity in America has often been defined in terms of the equalization of spending on 
students across local school districts and states, with courts intervening to order such 
equalization (Reed 2001). Eric Hanushek (1996, 1989), among others (e.g. Brandi 1998), has 
argued repeatedly that additional investments of resources in K-12 education, in general, fail to 
produce consistent payoffs in the form of higher student achievement. Other scholars have 
adopted the moderate position that more money improves educational outcomes, even 
dramatically, when it is invested properly (Hedges and Greenwald 1996, Hedges et al 1994). 
Such a claim could be fully consistent with Hanushek’ s findings, if it is the case that most 
schools do not invest additional resources in the sorts of products and activities that yield 
positive achievement benefits for students. The investment problem might be particularly acute 
for the iimer-city schools thattend to serve the populations who are eligible for vo uchers, as they 
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may face more claims for, less discretion regarding, and less expertise in the effective 
deployment of resources. 

B. Scale 

Throughout the Twentieth Century, a dominant maxim regarding K-12 education was 
that bigger is better. Large elementary and secondary schools could take advantage of 
economies of scale to provide better facilities and more diverse academic programs and extra- 
curricular activities to their larger student bodies. However, recently there has been a movement 
towards a “smaller is better” mentality. Smaller schools, it is thought, are less imposing to 
students and better able to provide them with individual attention (Nathan and Febey 2001). 
Smallness appears to be most important in the classroom, as two influential studies of a class- 
size reduction experiment in Tennessee determined that smaller classes enhance student 
performance, at least in the yoimger grades (Mosteller 1995; Krueger 1999). 

C. Commimitv 

Schools serve disadvantaged students best when they function not as educational 
bureaucracies or businesses but as educational communities. Noted education analysts James 
Coleman and Thomas Hoffer (1987), John Brandi (1998) and Anthony Bryk and his colleagues 
(1992) have all argued that students, especially those from disadvantaged backgroimds, thrive in 
private schools because of social capital that is produced by the nurturing of a strong sense of 
commimity. Private schools commimicate more with parents and draw them into the school to 
participate, in a partnership, in the education of their child. Teachers have a strong sense of 
mission that is driven by concerns for the well-being of each student. It is precisely this fostering 
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of education as a communal enterprise that results in disadvantaged students learning more in 
private schools, they argue. 

D. Order and Discipline 

Disadvantaged inner-city students are more likely to face disorder on a daily basis — in 
their neighborhoods, schools, and homes. The danger and unpredictability of chaotic 
environments naturally engenders fear and hopelessness in young people. Schools that are able 
to establish a safe and well-ordered environment, through strict discipline and careful monitoring 
of the school and classrooms, likely serve as a firebreak to hopelessness and fear for low-income 
urban children. Moreover, disorder in the classroom distracts teachers from teaching and 
students from learning. It is generally recognized that an orderly learning environment is a 
necessary pre-condition for effective learning in school (e.g. Wilson 1989, 21-23). 

E. Homework 

A munber of educational studies have demonstrated that students tend to live up to the 
high or low expectations that are set for them by parents and teachers. Ferguson (1998) argues 
that expectations are at least partly to blame for the infamous “black-white test score gap” of 
nearly two years in the achievement of comparable black and white students by the time that they 
are seniors in high schools. Less is expected of disadvantaged students in terms of educational 
achievement, and they tend to deliver only what is expected of them. If even disadvantaged 
students are assigned challenging homework and high expectations for success in completing it, 
progress might be made in closing the test score gap. 
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F. Peers 



A rich theoretical and empirical literature has established that characteristics of a 
student’s peer group have independent and significant effects on the student’s academic 
achievement (Nielsen and Wolf 2001; Hess and Leal 1997; Hoxby 2000b). Since the seminal 
Supreme Court ruling Brown v. Topeka Board of Education, our society has operated under the 
maxim that an educational system that is entirely separated by race is inherently unequal to the 
minority students involved. Thus, racial desegregation has long been thought to be an important 
vehicle for improving educational outcomes for low-income African American students. All 
other things equal, we would expect disadvantaged minority students to learn more if they were 
to be educated in a racially diverse environment. We also would expect lower-income students 
to perform better academically if they were educated with higher-income students as their peers. 

All of the above theories are possible candidates for explaining what happens inside the 
black box to produce voucher gains. However, even this extended list of theories fails to entirely 
exhaust the possible explanations. Another peer group factor, the average academic ability of the 
student body at the schools, could not be measured consistently, since different schools use 
different tests and performance standards. The leadership of school principals and the quality of 
teachers may be driving voucher gains. Since we were unable to obtain direct measures of those 
and other classroom factors, we may not be able to identify the proximate cause behind voucher 
gains. Still, we should be able to shed some light on the question. We now turn to the data we 
will analyze in order to preliminarily rule in or out each of these competing explanations. 
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III. Data AND Methods 



A. Data 

The core of the data that we use in the study comes from the second year evaluation of 
the Washington Scholarship Fund (WSF) privately-funded voucher program. The WSF provides 
partial tuition scholarships of up to $2,200 to families in the District of Coliunbia with household 
income at or below 270% of the federal poverty line. Families with income below the poverty 
line are eligible for the maximum scholarship amount, whereas families at 270% of poverty are 
eligible for about half of the maximum. The scholarships are like vouchers, which can be 
redeemed at any of the more than 100 D.C. private schools that participate in the scholarship 
program. WSF has been awarding scholarships to D.C. students since 1993. Currently, the WSF 
supports 1,325 elementary and secondary school students with scholarships. 

The WSF experienced a dramatic expansion of their scholarship program in 1998. Since 
demand greatly exceeded even the supply of 1,000 new scholarships, the vouchers were awarded 
by lottery. Because only the luck of the draw determined which family would or would not 
receive a voucher, the effect of the voucher on student and family outcomes could be studied via 
a Randomized Field Trial (RFT). In the spring of 1998, before the scholarships were awarded, 
the families of 1,582 students were surveyed about their family characteristics and school 
experiences. The students, who all were enrolled in grades 1-8 of a public school at the time, 
completed the Iowa Test of Basic Skills in reading and math in order to produce a baseline 
measure of their academic abilities. After baseline data collection was complete, 8 1 1 of the 
students in the study population were awarded scholarships by lottery. The remaining 771 
students comprised the control group for the study. The treatment and control groups in the D.C. 
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evaluation did not differ significantly on any of the nearly 50 characteristics (Wolf, Howell and 
Peterson 2000). The lottery appears to have worked. 

Two years after the voucher offer, in the spring of 2000, the remaining members of the 
treatment and control groups were invited to data collection sessions in which their children were 
re-tested and the parents and older students were again surveyed about their educational 
experiences. Since 1 25 members of the control group had won the turnout incentive lottery in 
the first re-test year of 1999, a total of 1,457 students remained in the study population. The 
second- year turnout of 730 students comprised 50% of that population, for both the treatment 
and control groups. 

The second-year WSF evaluation data were generated via a number of measurement 
instruments. The baseline and second-year test scores were generated by the performance of 
students on the ITBS in 1998 and 2000, respectively. Parental reports of school factors such as 
indicators of school community and order were obtained from survey instruments. The students 
in grades 4-9 were surveyed about such things as teacher attitudes, homework, and strict 
discipline at their schools (Wolf, Peterson and West 2001). 

These core data from the RFT were then supplemented by information collected from and 
about the various public and private schools that the students attended during the 1999-2000 
academic year. The supplemental data included information about per pupil spending on 
students, as well as statistics regarding student body characteristics, enrollments, and class sizes. 
For the non-charter D.C. public schools in the sample, these data were provided to us by the 
district’s Office of Public Accountability. For the private schools in the sample, the data were 
obtained from two sources, depending on the type of private school involved. Information about 
the Catholic parochial schools in the sample were obtained from the Office of the Superintendent 
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of Schools for the Catholic Archdiocese of Washington, D.C. Statistics regarding the non- 
parochial religious and independent private schools in the sample were obtained from responses 
to a survey we mailed to those schools. The response to the mail survey was high, thanks in part 
to our persistence in following up with private school administrators, as we obtained at least 
some information from over 80% of the private schools in the sample. 

Still, moderate data gaps remain, especially regarding potentially sensitive subjects such 
as the racial and income demographics of the schools. To prevent non-random list- wise deletion 
of observations from imperiling both the validity and efficiency of the analysis, we replaced 
missing data in our explanatory variables with 0 values and included missing data dummy 
variables in the analysis. By replacing the missing values, we avoided the inefficiency and 
potential bias caused by the exclusion of a significant proportion of the observations. By 
including a missing data dummy variable for any variable so altered we prevent the replacement 
zeroes from biasing the estimation of the beta coefficients in the statistical model (Cohen and 
Cohen 1983, pp. 275-300). Still, including a significant number of such non-theoretical 
covariates reduces the efficiency of the estimations somewhat (i.e. burning one degree of 
freedom per missing data variable) and potentialy biases estimates if the missing data dummies 
co-vary significantly with theoretically-based explanatory variables (Peterson and Howell 2003). 
To minimize the likelihood of such problems, each theoretical claim regarding the proximate 
causes of voucher gains was tested in a separate, parsimonious statistical model. 

Table A in the Appendix presents descriptive statistics regarding the variables used in the 
analysis. Where the operational measurement of the variable is not obvious, a description is 
provided in the notes to the table. As revealed by the numbers in the far right column, the most 
significant missing data problem comes from measures such as the “Caring Teaching Index” 
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which are constructed from student survey responses. That is because the third graders in the 
study were too young to be surveyed, and some of the students in the older grades who were 
surveyed left questions blank. Several of the variables that rely upon data drawn from the 
schools themselves also suffer from significant data gaps, especially those for student body 
demographics. Still, a reasonably large pool of actual data is available to estimate the 
relationships discussed in this paper. 

B. Methods 

In this paper, we report on the effects of private schooling on student test scores, also 
known as the effect of treatment on the treated. Even within the context of an RFT, such 
estimates can suffer from bias, since some of the participants who are randomly assigned to the 
treatment fail to use it (i.e. remain in public school) and some participants who are randomly 
denied the treatment obtain it anyway (i.e. enroll in private school without the aid of a voucher). 
In earlier studies, when the concern has been to obtain unbiased estimates of treatment impacts, 
evaluators used Instrumental Variable Analysis, with the lottery result as the exogenous 
instrument, in order to correct for such biases (Howell et al 2002). The relevant comparison 
here, though, is not between those students who attended private school and those in public 
schools who would have gone private if given the chance. Since we are seeking to identify the 
specific characteristics of schools that may be producing voucher gains, we need to use a 
straightforward private schooling treatment variable as the source of the effect that needs to be 
“explained away” by factors causally downstream of the switch to private school. 

To estimate the effect of private schooling on test scores, we run the following Ordinary 
Least Squares (OLS) model: 

Y^=a + P^P + ^ 
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is each student’s total achievement score after two years on the Iowa Test of Basic Skills 
expressed in NPR points.' The total achievement score is a simple average of the math and 
reading components.^ P is an indicator variable for whether or not an individual attended a 
private school during the second year of the evaluation. and Y^j^ are the baseline reading 

and math scores. Baseline test scores are included to adjust for minor differences between the 
treatment and control groups on achievement on the baseline tests, and to increase the precision 
of the estimated impacts. The /?, coefficient therefore represents the estimated impact of 
switching to private school on student test scores. This model does not include many of the 
individual-level demographic controls that are commonplace in the sector-effects literature. 
Elsewhere, researchers have shown that including demographic controls does not influence the 
estimates of the private schooling treatment effect in the D.C. evaluation (Howell et al 2002). 

To identify the effect of private schooling on study participants, we need only estimate 
the above equation. To identify the proximate causes of this effect, however, more complex 
analyses are required. In subsequent models, we include additional variables that describe the 
kinds of public and private schools that students attended in D.C. Should any one (or 
combination) of these factors explain voucher students’ test-score gains, their inclusion in the 
models should substantially attenuate the observed treatment effect. 

IV. Results 

The results of our initial search for a specifioexplanation for voucher achievement gains 
appear in Tables 1-7. The first row under the column headers in each table simply restates the 

’ For ease of interpretation, we report impacts in terms of NPR points. The results do not change substantively in 
any city when using National Curve Equivalents or raw test scores. 

^ Because it is based upon a larger number of test items, the total achievement score is likely to generate more 
stable estimates than are reading and math scores estimated separately (see Krueger 1999). 



average private schooling impact that is produced simply by estimating combined math and 
reading test score performance based on whether or not the student attended a private school, 
controlling only for each student’s baseline math and reading score. The average treatment gain 
of about 3.3 National Percentile Rank points, which is statistically significant even beyond 
p<.01, essentially serves as the target to knock down. Should that voucher benefit be largely the 
result of greater access to some particular educational advantage, the treatment effect coefficient 
should shrink precipitously in magnitudeand become indistinguishable from 0 upon the 
introduction of that factor or set of factors into the statistical model. 

Our quest begins with school resources. Do the participants in the WSF voucher program 
appear to be performing better academically largely because they have access to superior 
academic resources? Apparently, the answer is no. As the Model 1 results suggest, the gains are 
in no way limited to the small subset of study participants who attended elite private schools, 
defined as private schools thatare independent of any religious organization. The per-pupil 
spending level also does not explain the private schooling effect, as evidenced by the results of 
the Model 2 estimation. The expenditure variable itself is positive and statistically significant, as 
we might expect. However, controlling for per-pupil expenditures actually doubles the size of 
the voucher effect. This result suggests that the private schools in D.C. produced test score gains 
for study participants not because of higher expenditures but in spite o/’lower per-pupil spending 
than was common in the public schools in the sample. As demonstrated by the Model 3 
estimation, the voucher gains cannot be explained by access to more elaborate school facilities or 
programs. A significant voucher gain remains, and is even much larger in size, controlling for 
these three important school resource factors. 

[Table 1 about here] 
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If resources fail to explain away voucher gains, might the smaller scale of most private 
schools be their proximate cause? In this case, clearly the answer is no. As described in Table 2, 
the gain from the treatment again increases somewhat in magnitude when estimates of school 
size and class size are factored into the equation. Model 4 includes an explanatory variable for 
school enrollment, which is positive and statistically signifrcant, suggesting that larger schools 
actually enhanced the test-score performance of the students in the study. Model 5 adds school 
data regarding class size as an explanation for voucher gains, and the coefficient suggests that 
somewhat higher performance is associated with larger and not smaller classes, although the 
coefficient is not statistically significant. More importantly for our purposes, the coefficient for 
the treatment effect registers about 2 points higher than the threshold effect of 3.3 NPR in both 
estimations. The opportunity to be educated in smaller schools or classes does not seem to be 
driving voucher gains. The estimates for the constant term and the baseline test controls were 
omitted from this and subsequent results tables (but not the regression models that produced 
them) because they do not contribute to the substance of the analysis and the coefficients for the 
baseline control variables hardly vary in their magnitude or precision. 

[Table 2 about here] 

If the scale of the education that voucher students receive is not the source of the voucher 
achievement gains, then perhaps the students gain access to a more vibrant educational 
community of involved parents and caring teachers through vouchers. As demonstrated in Table 
3, these measures of community fail to explain fully the voucher gains. The index of school- 
parent communication and parental involvement fails to explain any variation in student 
achievement, either separately or combined with the Caring Teacher Index. Students who report 
that their teachers are particularly interested in and attentive to students do tend to achieve more. 
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and the private schooling effect loses some magnitude and one level of significance when that 
variable is included in the model; however, a sizable and statistically significant treatment effect 
remains even after controlling for the existence of more concerned teachers. The source of the 
voucher effect on test scores must largely lie elsewhere. 

[Table 3 about here] 

Table 4 presents the results from the examination of whether or not the more orderly 
environment and stricter discipline experienced by students in private schools explains their 
achievement gains. Although the introduction of both the order and discipline variables into 
Model 9 reduces the independent effect of private schooling somewhat, the private schooling 
variable remains the only statistically significant explanatory variable in the equation. We 
cannot conclude that a more orderly environment and stricter discipline explain voucher gains. 

[Table 4 about here] 

What about homework? Do voucher students tend to benefit from more frequent and 
demanding homework assignments? The results in Table 5 are suggestive that homework might 
be one source of the private schooling advantage uncovered in our evaluation. In the Model 10 
estimation, parental reports regarding the average amounts of daily homework are factored into 
the equation. More homework is associated with higher achievement downstream, but the 
relationship is not statistically significant. Moreover, the independent voucher effect sheds less 
than half a point and remains statistically significant. Because students are a more direct source 
of information about homework burdens. Model 1 1 factors their reports into the analysis. 
Although the treatment effect diminishes by more than a point, it remains statistically significant 
at least at the lowest level used in this analysis. Moreover, student estimates of their relative 
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amounts of daily homework are not statistically significant predictors of downstream 
achievement. 

[Table 5 about here] 

Do measurable characteristics of the peers with whom students are educated explain the 
D.C. voucher gains? Here we get our first whiff of a potential smoking gun. As presented in 
Table 6, the two measures of student body characteristics — ^percent white students and percent 
low-income students — demonstrate their expected effects on student achievement. The study 
students who are in schools with higher percentages of white students tend to perform somewhat 
better on the follow-up achievement test, all else being equal; whereas, students who are in 
schools with higher percentages of low-income students may tend to perform somewhat less 
well, but the effect is statistically indistinguishable from 0. Controlling for these student body 
characteristics simultaneously reduces the magnitude and precision of the treatment effect by 
nearly a full point, for the first time rendering it statistically insignificant, although barely so. It 
appears that exposure to a more advantaged group of peers may be one important cause of the 
voucher gains that we uncovered in D.C. 

[Table 6 about here] 

Finally, the possible factors considered here may not, separately, explain voucher gains; 
however, several of them may provide an adequate explanation for such gains when considered 
together. Table 7 presents the results of a model estimation that includes factors from multiple 
schools of thought regarding the drivers of voucher gains that have individually demonstrated 
promise as possible proximate causes of D.C. voucher gains. Model 14 finally accomplishes 
what the previous estimations could not: the school characteristic variables in the model shrink 
the magnitude of the independent private schooling effect by nearly 40% and render it 
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statistically insignificant clearly beyond conventional confidence levels (p=.15). The higher 
percentage of white students at D.C. private schools and the dedication of the private school 
teachers appear to be the primary drivers of the voucher gains that we uncovered. However, the 
model is highly sensitive to specification changes. If the percent low-income and amoimt of 
homework variables (or the controls for baseline reading and math scores) are omitted Irom the 
estimation, the independent private schooling effect remains sizable and statistically significant 
even controlling for the percentage of white students and the caring teacher index. 

V. Conclusion 

Just as Rome was not built in a day, the answer to the vexing question of what school 
characteristics or practices might explain the gains evidenced by voucher users is not likely to 
emerge, decisively, Irom a preliminary analysis of data from a single voucher experiment. It is 
left to future analyses to generate more conclusive results regarding the proximate causes of 
voucher gains. The analysts who conduct such probes should especially target student body 
demographics and measures of the relative dedication of teachers in their search for why low- 
income inner-city students appear to benefit academically from private schooling. The amoimt 
and difficulty of homework also has emerged from this study as a possible component of the 
explanation for such gains in D.C. For now, we should draw only tentative conclusions 
regarding what happens inside the black box to generate voucher gains. We also should 
encourage more analyses that peer into future voucher black boxes, hopefully with a stronger 
microscope. 
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Table 1: Do School Resources Explain Voucher Gains? 



Factor 


Base Model 


Model 1 


Model 2 


Model 3 


Voucher Gain Alone 


3.27*** 


3.27*** 


3.27*** 


3.27*** 




(1.25) 


(1.25) 


(1.25) 


(1.25) 


Explanatory Variables 










Private School Effect 


3.27*** 


3.15** 


6.51*** 


6.46*** 




{l.2f 


(1.2J1 


(2.23) 


(2.26) 


Elite Private Schooling 




1.60 


-.58 


-.50 






(3.9? 


(4.5J) 


(4.5) 


Per Pupil Expenditure 






.82* 


.82* 


(in thousands) 






(.5$) 


(.50) 


School Facility Index 








-.02 










(.2J) 


Control Variables 










Baseline Reading Score 


.13*** 


.13*** 


.12*** 


.12*** 




(.02) 


(.02) 


(.02) 


(.02) 


Baseline Math Score 


.43*** 


.43*** 


.42*** 


.42*** 




(.03) 


(.03) 


(.03) 


(.03) 


Constant 


7.96*** 


7.96*** 


2.70 


2.88 




(1.09) 


(1.09) 


(3.4J) 


(3.80) 


Adjusted 


.36 


.36 


.37 


.37 


N 


730 


730 


730 


730 



Observations are of students in grades 3-9 in 2000. Observations weighted to correct for non- 
response. Missing data dummy variables included in all regressions. Figures are unstandardized 
regression coefficients with standard errors in parentheses below. * denotes statistical significance 
beyond p<.10, ♦♦ p<.05, ♦♦♦ p<.01, using a two-tailed test. 
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Table 2: Does the Scale of the Educational Environment Explain 
Voucher Gains? 



Factor 


Model 4 


Model 5 


Voucher Gain Alone 


3.27*** 


3.27*** 




(1.25) 


(1.25) 


Explanatory Variables 






Private School Effect 


5.52*** 


5.24*** 




(1.50) 


(1.55) 


School Enrollment 


1.16*** 


1.12*** 


(in 00s) 


(.39) 


(.39) 


Class Size 




.19 






(.13) 




.37 


.37 


N 


730 


730 



Observations are of students in grades 3-9 in 2000. Observations weighted to 
correct for non-response. Missing data dummy variables, constant, and controls 
for reading and math baseline scores included in all regressions but coefficients 
omitted from tables. Figures are unstandardized regression coefficients with 
standard errors in parentheses below. ^ denotes statistical significance beyond 
p<.10, ** p<.05, *** p<.01, using a two-tailed test. 

Table 3: Do Stronger Communities Explain Voucher Gains? 



Factor 


Model 6 


Model 7 


Voucher Gain Alone 


3.27*** 


3.27*** 




(1.25) 


(1.25) 


Explanatory Variables 






Private School Effect 


3.30*** 


2 . 75 ** 




(1.2? 


(1.2) 


Community Index 


-.04 


-.03 




(.34) 


(.32) 


Caring Teacher Index 




.74*** 

(.25) 




.36 


.43 


N 


730 


730 



Observations are of students in grades 3-9 in 2000. Observations weighted to 
correct for non-response. Missing data dummy variables, constant, and controls 
for reading and math baseline scores included in all regressions but coefficients 
omitted from tables. Figures are unstandardized regression coefficients with 
standard errors in parentheses below. * denotes statistical significance beyond 
p<.10, ** p<.05, *** p<.01, using a two-tailed test. 
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Table 4: Do Order and Discipline Explain Voucher Gains? 



Factor 


Model 8 


Model 9 


Voucher Gain Alone 


3.27*** 


3.27*** 




(1.25) 


(1.25) 


Explanatory Variables 






Private School Effect 


3.18** 


2.41** 




{1.2} 


(l.ljl 


Order Index 


.19 


.14 




(.12) 


(.11) 


Strict Discipline 




.88 

(.73) 




.37 


.44 


N 


730 


730 



Observations are of students in grades 3-9 in 2000. Observations weighted to 
correct for non-response. Missing data dummy variables, constant, and controls 
for reading and math baseline scores included in all regressions but coefficients 
omitted from tables. Figures are unstandardized regression coefficients with 
standard errors in parentheses below. * denotes statistical significance beyond 
p<.10, ** p<.05, *** p<.01, using a two-tailed test. 
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Table 5; Does Homework Explain Voucher Gains? 



Factor 


Model 10 


Modem 


Voucher Gain Alone 


3.27*** 


3.27*** 




(1.25) 


(1.25) 


Explanatory Variables 






Private School Effect 


2.88^^ 


2.16^ 




(1.2J1 


(1.2? 


Homework Half-hours 


.97 




(Parent Reports) 


(.63) 




Homework Half-hours 




.67 


(Student Reports) 




(.4^ 




.37 


.42 


N 


730 


730 



Observations are of students in grades 3-9 in 2000. Observations weighted to 
correct for non-response. Missing data dummy variables, constant, and controls 
for reading and math baseline scores included in all regressions but coefficients 
omitted from tables. Figures are unstandardized regression coefficients with 
standard errors in parentheses below. * denotes statistical significance beyond 
p<.10, ♦♦ p<.05, ♦♦♦ p<.01, using a two-tailed test. 

Table 6: Do Student Body Demographics Explain Voucher 
Gains? 



Factor 


Model 12 


Model 13 


Voucher Gain Alone 


3.27*** 


3.27*** 




(1.25) 


(1.25) 


Explanatory Variables 






Private School Effect 


3.02*^ 


2.32® 




(1.26) 


(1.45) 


Percent White Students 


.18^^^ 


.15** 




(.06) 


(.06) 


Percent Low-Income 




-.03 






(.03) 




.37 


.37 


N 


730 


730 



Observations are of students in grades 3-9 in 2000. Observations weighted to 
correct for non-response. Missing data dummy variables, constant, and controls 
for reading and math baseline scores included in all regressions but coefficients 
omitted from tables. Figures are unstandardized regression coefficients with 
standard errors in parentheses below. ^ denotes statistical significance beyond 
p<.10, ♦♦ p<.05, ♦♦♦ p<.01, using a two-tailed test. 

®p=. Ill 
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Table 7: Do School Demographics, Homework, and 



Factor 


Model 14 


Voucher Gain Alone 


3.27*** 

(1.25) 


Explanatory Variables 




Private School Effect 


2.00 




(\39 


Percent White Students 






(.06) 


Percent Low-Income 


-.00 




(.03) 


Homework Half-hours 


.56 


(Student Reports) 


(.43) 


Caring Teacher Index 


72 ^^^ 




(.25) 




.44 


N 


730 



Observations are of students in grades 3-9 in 2000. Observations 
weighted to correct for non-response. Missing data dummy 
variables, constant, and controls for reading and math baseline 
scores included in all regressions but coefficients omitted from 
tables. Figures are unstandardized regression coefficients with 
standard errors in parentheses below. ^ denotes statistical 
significance beyond p<.10, ♦♦ p<.05, p<.01, using a two-tailed 

test. 
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Appendix for Variables Used in the 
Analysis 



Table A: Descriptive Statistics 


of Variables for DC Stud 


ly Participants in 2000 


Factor 


Mean 


Standard 

Deviation 


Minimum 


Maximum 


Valid N 


Dependent Variable: 












Combined Reading & 


22.9 NPR' 


19.3 NPR 


ONPR 


92.5 NPR 


730 


Math Score in 2000 
Explanatory Variables: 












Private School 


31.4% 


46.4% 


0 


1 


730 


Elite Private School^ 


2 .5% 


15.6% 


0 


1 


726 


Per Pupil Expenditure^ 


$5,824 


$2,071 


$395 


$18,000 


697 


School Facility Index^ 


9.3 


3.1 


1 


14 


704 


White Students 


3.5% 


12.9% 


0% 


81% 


482 


Free/Reduced Lunch 


68.4% 


28.8% 


0% 


99% 


448 


School Enrollment 


389.9 


205.1 


37 


1543 


527 


Class Size 


17.3 


5.8 


4.4 


56.7 


480 


Community Index^ 


5.9 


1.8 


1 


8 


698 


Caring Teacher Index^ 


15.7 


2.8 


6 


20 


439 


School Order Index’ 


17.7 


4.7 


7 


21 


730 


Strict Discipline^ 


2.9 


.9 


1 


4 


459 


Parent Est. Homework 


71 min. 


28 min. 


0 min. 


150 min. 


693 


Student Est. Homework 


85 min. 


45 min. 


15 min. 


165 min. 


527 


Control Variables: 












Baseline Reading Score 


30.3 NPR 


27.1 NPR 


ONPR 


99 NPR 


730 


Baseline Math Score 


23.1 NPR 


21.8 NPR 


ONPR 


99 NPR 


730 



Restricted to students in grades 3-9 in 2000 



* National Percentile Ranks 

^ Defined as private but unaffiliated with a religious organization 

^ Defined as the regular tuition level at private schools and the district per-pupil expenditure level (net of special 
education) for public schools, differentiated by neighborhood public and public charter 

^ Based on parental responses regarding the presence or absence of: a computer lab, a library, a gym, a cafeteria, 
special programs for non-English speakers, individual tutors, special programs for slow learners, special programs 
for fast learners, child counselors, a nurse’s office, a music program, an arts program, an after-school program, and 
prepared lunches 

^ Based on parental responses to questions regarding whether or not they are: sent mid-term grades, notified when 
student is disruptive, asked to talk to class about their job, asked to assist in instruction, asked to attend open-houses 
at school, asked to attend parent/teacher conferences, sent regular notes from the teacher about student, sent a school 
newsletter 

^ Based on student Likert-scale responses to whether they agree that their teachers: are interested in students, really 
listen to them, are fair, avoid putting down students, punish cheating when they observe it 

’ Based on parental reports reporting the relative seriousness (very, somewhat, not) of the following disorders: kids 
destroying property, tardiness, truancy, kids fighting, kids cheating, racial conflict, and guns or other weapons 
® Based on student Likert-scale responses 
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